Long Branch
How Do LLMs Use Their Depth?
Gupta, Akshat, Yeung, Jay, Anumanchipalli, Gopala, Ivanova, Anna
Growing evidence suggests that large language models do not use their depth uniformly, yet we still lack a fine-grained understanding of their layer-wise prediction dynamics. In this paper, we trace the intermediate representations of several open-weight models during inference and reveal a structured and nuanced use of depth. Specifically, we propose a "Guess-then-Refine" framework that explains how LLMs internally structure their computations to make predictions. We first show that the top-ranked predictions in early LLM layers are composed primarily of high-frequency tokens, which act as statistical guesses proposed by the model early on due to the lack of appropriate contextual information. As contextual information develops deeper into the model, these initial guesses get refined into contextually appropriate tokens. Even high-frequency token predictions from early layers get refined > 70% of the time, indicating that correct token prediction is not "one-and-done". We then go beyond frequency-based prediction to examine the dynamic usage of layer depth across three case studies. Together, our results provide a detailed view of depth usage in LLMs, shedding light on the layer-by-layer computations that underlie successful predictions and providing insights for future works to improve computational efficiency in transformer-based models. Despite the remarkable performance of large language models (LLMs), their internal computations remain poorly understood. One critical question is: how do LLMs internally structure their computations during inference and use their depth layer-by-layer to arrive at predictions? Are specific token predictions always computed at the last layer or does the model settle on predictable tokens early on and simply propagate these predictions? These questions have implications both for interpreting the internal computations of these models and for building more efficient LLM that can use their compute dynamically.
- North America > United States > New Jersey > Monmouth County > Long Branch (0.04)
- North America > United States > California > Alameda County > Berkeley (0.04)
The Evolution and Future Perspectives of Artificial Intelligence Generated Content
Zhu, Chengzhang, Cui, Luobin, Tang, Ying, Wang, Jiacun
Artificial intelligence generated content (AIGC), a rapidly advancing technology, is transforming content creation across domains, such as text, images, audio, and video. Its growing potential has attracted more and more researchers and investors to explore and expand its possibilities. This review traces AIGC's evolution through four developmental milestones-ranging from early rule-based systems to modern transfer learning models-within a unified framework that highlights how each milestone contributes uniquely to content generation. In particular, the paper employs a common example across all milestones to illustrate the capabilities and limitations of methods within each phase, providing a consistent evaluation of AIGC methodologies and their development. Furthermore, this paper addresses critical challenges associated with AIGC and proposes actionable strategies to mitigate them. This study aims to guide researchers and practitioners in selecting and optimizing AIGC models to enhance the quality and efficiency of content creation across diverse domains.
- North America > United States > New Jersey > Gloucester County > Glassboro (0.14)
- North America > United States > New Jersey > Monmouth County > Long Branch (0.14)
- Asia > Singapore (0.04)
- (10 more...)
- Research Report (1.00)
- Overview (1.00)
- Information Technology > Security & Privacy (1.00)
- Health & Medicine (1.00)
- Education (1.00)
- (2 more...)
Transaction Fraud Detection via an Adaptive Graph Neural Network
Tian, Yue, Liu, Guanjun, Wang, Jiacun, Zhou, Mengchu
Many machine learning methods have been proposed to achieve accurate transaction fraud detection, which is essential to the financial security of individuals and banks. However, most existing methods leverage original features only or require manual feature engineering. They lack the ability to learn discriminative representations from transaction data. Moreover, criminals often commit fraud by imitating cardholders' behaviors, which causes the poor performance of existing detection models. In this paper, we propose an Adaptive Sampling and Aggregation-based Graph Neural Network (ASA-GNN) that learns discriminative representations to improve the performance of transaction fraud detection. A neighbor sampling strategy is performed to filter noisy nodes and supplement information for fraudulent nodes. Specifically, we leverage cosine similarity and edge weights to adaptively select neighbors with similar behavior patterns for target nodes and then find multi-hop neighbors for fraudulent nodes. A neighbor diversity metric is designed by calculating the entropy among neighbors to tackle the camouflage issue of fraudsters and explicitly alleviate the over-smoothing phenomena. Extensive experiments on three real financial datasets demonstrate that the proposed method ASA-GNN outperforms state-of-the-art ones.
- North America > United States > New Jersey > Monmouth County > Long Branch (0.04)
- North America > United States > New Jersey > Essex County > Newark (0.04)
- Asia > China > Shanghai > Shanghai (0.04)
Richard Anderson, costar of 'The Six MIllion Dollar Man' and 'The Bionic Woman,' dies at 91
Richard Anderson, the tall, handsome actor best known for costarring simultaneously in the popular 1970s television shows "The Six Million Dollar Man" and "The Bionic Woman," has died at age 91. Anderson died of natural causes on Thursday, family spokesman Jonathan Taylor said. "The Six Million Dollar Man" brought a new wave of supernatural heroes to television. Based on the novel "Cyborg" by Martin Caidin, it starred Lee Majors as U.S. astronaut Steve Austin, who is severely injured in a crash. The government saves his life by rebuilding his body with atomic-powered artificial limbs and other parts, giving him superhuman strength, speed and other powers.
- North America > United States > California > Los Angeles County > Los Angeles (0.07)
- North America > United States > New Jersey > Monmouth County > Long Branch (0.05)
- North America > United States > Missouri (0.05)
- Leisure & Entertainment (1.00)
- Health & Medicine > Health Care Technology (0.85)
- Government > Military > Army (0.34)
‘$6 Million Man’ actor dies
Richard Anderson, the tall, handsome actor best known for costarring simultaneously in the popular 1970s television shows "The Six Million Dollar Man" and "The Bionic Woman," has died at age 91. Anderson died of natural causes on Thursday, family spokesman Jonathan Taylor told The Associated Press. "The Six Million Dollar Man" brought a new wave of supernatural heroes to television. Based on the novel "Cyborg" by Martin Caidin, it starred Lee Majors as U.S. astronaut Steve Austin, who is severely injured in a crash. The government saves his life by rebuilding his body with atom-powered artificial limbs and other parts, giving him superhuman strength, speed and other powers.
- North America > United States > California > Los Angeles County > Los Angeles (0.07)
- North America > United States > New Jersey > Monmouth County > Long Branch (0.05)
- North America > United States > Missouri (0.05)
- Leisure & Entertainment (1.00)
- Health & Medicine > Health Care Technology (0.86)
- Government > Military > Army (0.31)
Inside Ford's Top-Secret Campaign to Remake the Iconic GT Supercar
It isn't often that one car completely dominates the conversation at a major international auto show. And it isn't often that one car so completely symbolizes a company's return from the brink of ruin. But that exact confluence happened in January 2015 in Detroit, at the North American International Auto Show, the biggest car show of them all. The unveiling of Ford's new GT supercar was the culmination of a year of tantalizing rumors, which had begun to take shape in the fall of 2014 and then built momentum. The speculation went something like this: with 2016 right around the corner, the Ford Motor Company was seriously contemplating a return to what practically everyone in racing considers the automaker's moment of purest glory on the track, Le Mans in '66. Everyone did the math: 2016 minus 1966 was fifty years.
- North America > United States > New Jersey > Monmouth County > Long Branch (0.04)
- North America > United States > Illinois > Cook County > Chicago (0.04)
- Leisure & Entertainment > Sports > Motorsports > Sports Car Racing (1.00)
- Automobiles & Trucks > Manufacturer (1.00)
Support Vector Regression Machines
Drucker, Harris, Burges, Christopher J. C., Kaufman, Linda, Smola, Alex J., Vapnik, Vladimir
A new regression technique based on Vapnik's concept of support vectors is introduced. We compare support vector regression (SVR) with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimensionality of the input space.
- North America > United States > California > Alameda County > Berkeley (0.14)
- North America > United States > New Jersey > Monmouth County > Long Branch (0.04)
Support Vector Regression Machines
Drucker, Harris, Burges, Christopher J. C., Kaufman, Linda, Smola, Alex J., Vapnik, Vladimir
A new regression technique based on Vapnik's concept of support vectors is introduced. We compare support vector regression (SVR) with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimensionality of the input space.
- North America > United States > California > Alameda County > Berkeley (0.14)
- North America > United States > New Jersey > Monmouth County > Long Branch (0.04)
Support Vector Regression Machines
Drucker, Harris, Burges, Christopher J. C., Kaufman, Linda, Smola, Alex J., Vapnik, Vladimir
A new regression technique based on Vapnik's concept of support vectors is introduced. We compare support vector regression (SVR) with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimensionality of the input space.
- North America > United States > California > Alameda County > Berkeley (0.14)
- North America > United States > New Jersey > Monmouth County > Long Branch (0.04)